Page 1
arXiv:quant-ph/0305062v2 17 Feb 2005 R´ENYI EXTRAPOLATION OF SHANNON ENTROPY KAROL ˙ZYCZKOWSKI Abstract. Relations between Shannon entropy and R´enyi entropies of integer order are discussed. For any N–point discrete probability distribution for which the R´enyi entropies of order two and three are known, we provide an lower and an upper bound for the Shannon entropy. The average of both bounds provide an explicit extrapolation for this quantity. These results imply relations between the von Neumann entropy of a mixed quantum state, its linear entropy and traces. ver. 2 with corrigendum added, February 17, 2005
- Introduction We are going to analyze discrete probability distributions ⃗x = {x1, ..xN}, which consist of non–negative numbers summing to unity. PN i=1 xi = 1. To characterize quantitatively such probability vectors one uses Shannon (information) entropy [1] H(⃗x) := − N X i=1 xi ln xi (1) where we adopt the convention that 0 ln 0 = 0, if necessary. The Shannon entropy is distinguished by several unique properties [2], but it is often convenient to introduce generalized R´enyi entropies [3] parametrized by a continuous parameter q, Hq(⃗x) := 1 1 −q ln h N X i=1 xq i i . (2) The R´enyi entropies are well defined for positive q ̸= 1, but is is not difficult to show that for any probability distribution ⃗x one has limq→1 Hq(⃗x) = H(⃗x). For consistency the Shannon entropy H will thus be denoted by H1. This method of generalizing the Shannon entropy is by far not the only one – for reviews of other generalizations see books by Kapur [4] and Arndt [5]. In this work we discuss relations between R´enyi entropies of different orders, and in particular between H1, H2 and H3. Physical motivation for such a study is twofold. First, we may not know the entire vector ⃗x, but only a few of its Lq norms, so knowing the Renyi entropies, say H2 and H3 we want to estimate the unknown Shannon entropy H1. Such a situation occurs if one studies an N dimensional quantum mechanical mixed state ρ according to the scheme recently proposed by P. Horodecki et al. [6, 7] and measures directly the traces Trρk for k = 2, 3, …M. If M < N the entire spectrum of ρ remains unknown, and it is not possible to find its von Neumann entropy, S(ρ) = −Trρ ln ρ, (i.e. the Shannon entropy of the spectrum), but the generalized Renyi entropies Hk, including the linear entropy, which is a function of H2 may be readily obtained. Similar problems arise in many 1
Page 2
2 K. ˙ZYCZKOWSKI different branches of physics, for instance by the study of the statistics between particles created in high–energy collisions [8, 9]. Measuring probabilities of that two independent collisions give rise to the same distribution of particles allows one to obtain the R´enyi entropy H2, but not directly the Shannon entropy H1. Another reason to study relations between H1 and H2 has been provided by the work by Pipek and Varga [10]. They assumed that both these quantities are known, and observed that their difference, Sstr := H1 −H2 called structural en- tropy, provides an important characterization of the analyzed probability vector ⃗x. For instance, an increase of the structural entropy charactering an eigenstate of a tight binding model indicates the Anderson transition. Several other applications of structural entropy include also quantum chemistry and statistical analysis of quantum spectra, (see [11] and references therein). The von Neumann entropy of a mixed state obtained by partial trace of a bi– partite pure state, ρ = TrB(|ψ⟩⟨ψ|), measures the degree of entanglement of the pure state |ψ⟩. Alternatively one can measure the entanglement by generalized en- tropies (see e.g. [12]), so relations between entropies analyzed in this work provide bounds between different measures of entanglement. This very point has recently been discussed in the paper by Wei et al. [13], which provides an additional moti- vation for the present work. This paper is ogranized as follows. In section 2 the basic properties of the R´enyi entropies are reviewed. In section 3 we present recent results of Topsøe and Harremo¨es [14, 15], which allow us to propose lower and upper bounds on the Shannon entropy obtained out of the R´enyi entropies of order two and three, provided the length N of the vector is known. They are derived in section 4, while in section 5 we propose and analyze en estimation of the Shannon entropy. 2. Shannon and R´enyi entropies Consider a random variable ξ attaining not more than N different values with probabilities xi, i = 1, … , N. Such discrete probability distribution P may be cahracterized by the Shannon entropy (1) or generalized R´enyi entropies (2). All generalized entropies Hq vary from zero for a certain event (the distri- bution Q1 := {1, 0, …, 0}) to ln N, for the uniform distribution, (the distribu- tion QN := {1/N, 1/N, …, 1/N}). For the distributions with k equal elements, Qk := {1/k, …, 1/k, 0, …, 0}, the entropies admit intermediate values, ln k. The R´enyi entropy Hq converges to the Shannon entropy in the limit q →1. It is also useful to express the Shannon entropy as the limit of the derivative, H(⃗x) = −lim q→1 ∂[(1 −q) exp �Hq(⃗x) ] ∂q . (3) Some special cases of Hq are of special interest. For q = 2 we have H2(⃗x) = −ln[PN i=1 x2 i ]. The R´enyi entropy of order two, called extension entropy [10], is closely related to the inverse participation ratio, R(⃗x) := 1 PN i=1 x2 i = exp[H2(⃗x)]. (4) This quantity characterizes the ”effective number of different events” which the stochastic variable may admit, and varies from unity for Q1, to N for the uniform distribution QN. Another quantity r = 1/R = PN i=1 x2 i , called index of coincidence [15], in quantum mechanical problems is called purity, since the larger r the more
Page 3
R´ENYI EXTRAPOLATION OF SHANNON ENTROPY 3 pure, the state it describes. The quantity L := 1 −r = 1 −exp(−H2) is called linear entropy since in analogy to Shannon entropy it achieves its maximum for the uniform distribution QN. In the case q = 0 the R´enyi entropy is a function of the number m of positive components of the vector, H0(⃗x) = ln m. In the limit q →∞we obtain a quantity analogous to the Chebyshev norm: H∞= −ln xmax, where xmax is the largest component of ⃗x. The R´enyi entropy (2) is a sum of N terms so for any finite N the function of Hq on q is differentiable. The functional dependence of the R´enyi entropy on its parameter was investigated in detail by Back and Schl¨ogl [16]). Making use of the fact that the function xs is convex for s > 1 and concave for 0 ≤s ≤1 they have proved several inequalities1, which we recall in the case q > 0, ∂ ∂qHq ≤0, (5) ∂ ∂q q −1 q Hq ≥0, (6) ∂ ∂q (1 −q)Hq ≤0, (7) ∂2 ∂q2 (1 −q)Hq ≥0. (8) The first inequality (5) means that the R´enyi entropy is a non increasing function of its parameter, Hq(⃗x) ≥Hs(⃗x) for any s > q (9) and this statement is valid also for infinite probability vectors and the cases of nondifferentiable Hq [16]. Hence the structural entropy Sstr := H1 −H2 is non– negative [10]. Inequality (8) implies that the dependence of the R´enyi entropy on its parameter is convex.2 Thus knowing the 0 and 2–entropies one obtains by linear interpolation an upper bound for the Shannon entropy H1(⃗x) ≤Hu0 := 1 2(H0(⃗x) + H2(⃗x)). (10) This relation gives us an upper bound for the structural entropy Sstr(⃗x) := H1(⃗x) −H2(⃗x) ≤1 2 �H0(⃗x) −H2(⃗x) (11) valid for any vector ⃗x of a finite length N. In an analogous way, if the R´enyi entropies of order 2 and 3 are known, the linear extrapolation provides a lower bound for the Shannon entropy H1(⃗x) ≥Hd23(⃗x) = 2H2(⃗x) −H3(⃗x), (12) 1In the book[16] the quantity −Hq called R´enyi information was analyzed, so the direction of the inequalities derived there is inverted. 2This claim is withdrawn - see into corrigendum Sec. 7, in which consequences of this error are pointed out.
Page 4
4 K. ˙ZYCZKOWSKI which combined with (10), allows one to write down a simple estimation H023 = (Hd + Hu0)/2, (see Fig 3.a), H1(⃗x) ≈H023(⃗x) := 1 4[H0(⃗x) + 5H2(⃗x) −2H3(⃗x)]. (13) Making use of the inequality (6) we obtain the relation q −1 q Hq(⃗x) ≤s −1 s Hs(⃗x) for any s ≥q , (14) which is equivalent to the statement that the Lq–norm is a non–increasing function, ||⃗x||s ≤||⃗x||q. This result provides another upper bound, Hq ≤q(s−1)Hs/s(q −1). Although it is not applicable for the Shannon entropy, for which q = 1 so the inequality becomes trivial, but it gives an usefull bound on Hq with g > 1 by the limiting value H∞, Hq(⃗x) ≤ q q −1H∞(⃗x). (15) In further sections of this work we shall discuss possibilities of finding more precise bounds and estimations for the Shannon entropy, provided the dimension N of the probability vector is known. 3. Bounds between R´enyi entropies For any value q ≥0 the generalized entropy Hq is equal zero for certain events described by the distribution Q1, and achieves its maximum for the uniform distri- bution, S(QN) = ln N. To investigate further relations between the R´enyi entropies of different order we have chosen to analyze the case of N = 3 dimensional vectors ⃗x. The space of all possible probability vectors, plotted in the the plane x3 = 1 −x1 −x2 forms an equilateral triangle of side √ 2 measured in the Euclidean distance. Its three corners: (100), (010) and (001) represent certain events, while the center of the triangle corresponds to the uniform distribution Q3. Fig. 1 shows sets of points characterized by the same R´enyi entropy of order q, which may be called iso-entropy curves. Independently of the value of the parameter q the generalized entropy attains its minimum, Hq = 0, at the corners of the triangle, while the maximum Hq = ln 3 is achieved at the point Q3 at the center of the triangle. As shown in Fig. 1a the maximum is rather flat for q = 1/4. The case shown in this panel resembles the limiting case H0, for which the entropy reflects the number of events which may occur: it vanish at the corners of the triangle, is equal to ln 2 at its sides and equals to ln 3 for any point inside the triangle. The other example, q = 8, presented in Fig. 1d. is similar to the limiting case H∞, for which the iso-entropy curves are perpendicular to the lines joining Q3 with the corners. Superimposing some of the above pictures on one graph allows one to understand further relations between the R´enyi entropies. The generalized entropies are corre- lated; e.g. for the distributions Qk the entropies are equal to ln k independently of the value of q. The problem, which values the entropy Hq may admit, provided Hs is given, has been solved by Harremo¨es and Topsøe [15]. For any distribution P ∈RN they proved a simple (but not very sharp) upper bound on H1 by H2, H2(⃗x) ≤H1(⃗x) ≤ln N + 1/N −exp(−H2(⃗x)) (16)
Page 5
R´ENYI EXTRAPOLATION OF SHANNON ENTROPY 5 a) q=1/4 (100) (010) (001) * b) q=1 (100) (010) (001) * Q 1 Q 2 Q 3 c) q=2 (100) (010) (001) * Q 1 Q 2 Q 3 d) q=8 (100) (010) (001) * Figure 1. Iso-entropy curves in the space of probability distribu- tions with N = 3. The generalized R´enyi entropy is constant along the curves plotted for (a) q = 1/4, (b) q = 1 (Shannon entropy), (c) q = 2 (Euclidean circles–distance D2), and (d) q = 8. Dotted lines form the triangle △(Q1, Q2, Q3). where the lower bound is a special case of (9). Moreover, they showed that the set ∆q,s of possible probability distributions plotted in the plane Hq versus Hs is not convex (see Fig. 2), and its boundaries are formed of arcs corresponding to the interpolating probability distributions Qk,l(a) := aQk + (1 −a)Ql with a ∈[0, 1]. (17) More precisely, for any probability distribution P consisting of N components and arbitrary s > q > 0 the following bounds hold [15] Hq(Qk−1,k(a)) ≤Hq(P) ≤Hq(Q1,N(a)), (18) where a is a function of the known value of Hs(P) and the natural number k is selected by the inequality ln(k −1) ≤Hs(P) ≤ln k. The above results, crucial for the main body of this work, are easy to understand. Let us discuss the simplest nontrivial case with N = 3. The two dimensional simplex of probability distributions may be divided into 6 identical parts, equivalent to the triangle △(Q1, Q2, Q3), as shown in Fig. 1c. Three sides of the triangle are formed of the interpolating distributions Q1,2, Q1,3 and Q2,3 and these distinguished probabability distributions are extreme in a sense that they lead to the bounds (18). The bounds between H1 and H2 for N = 3 are presented in Fig. 2a. To obtain them it is sufficient to travel along the sides of the triangle △(Q1, Q2, Q3), computing H1 and H2 at each point and to plot the data obtained in the plane H1 versus H2. More formally, the upper boundary of the set ∆q,s consist of one arc derived from the family of distributions Q1,N(a); for any value of a we compute Hs(a), invert it to obtain a(Hs) and plot Hq(a(Hs)). In the case N = 3 the upper bounds plotted
Page 6
6 K. ˙ZYCZKOWSKI in Fig. 2a, c and e arise from the hypotenuse Q1, Q3 of the triangle △(Q1, Q2, Q3) from Fig. 1.b and c. In the similar way the lower bound may be derived from the distributions Qk,k+1(a) for k = 1, …N −1. It consists of (N −1) arcs forming an cascade Hq(Q1,2(a)) ⌣Hq(Q2,3(a)) ⌣· · · ⌣Hq(QN−1,N(a)). Note that the distributions Qk are represented in each plot by the points (ln k, ln k), which connect the neigh- bouring arcs. For N = 3 the lower bound consists of two arcs, corresponding to the adjacent sides Q1, Q2 and Q2, Q3 of △(Q1, Q2, Q3). The shape of the set ∆q,s requires a comment. The N −1 dimensional simplex – the set of all N–points probability distributions is convex and any of its projections onto a plane forms a convex set. However, its image at the plane Hq versus Hs needs not to be convex, since the transformations Hq(⃗x) and Hs(⃗x) are nonlinear. The boundaries of ∆q,s are obtained as the image of an appropriately chosen path on the boundary of the simplex. In the case considered it is the path Q1 →Q2 → · · · →QN →Q1, independently of the values of q and s. Observe that the general structure of the set ∆q,s does not depend on s. However, the larger difference s−q, the larger area of the set: the less information on Hq is provided by Hs. 0 0.5 1 0 0.5 1 N=3 a) H1 H 2 Q 1 Q 2 Q 3 0 0.5 1 0 0.5 1 c) H1 H 3 Q 1 Q 2 Q 3 0 0.5 1 0 0.5 1 e) H1 H4 Q 1 Q 2 Q 3 0 1 0 0.5 1 1.5 N=5 b) H1 H2 Q 1 Q 2 Q 3 Q 4 Q 5 0 1 0 0.5 1 1.5 d) H1 H3 Q 1 Q 2 Q 3 Q 4 Q 5 0 1 0 0.5 1 1.5 f) H1 H4 Q1 Q 2 Q 3 Q 4 Q 5 Figure 2. The set of all possible discrete distributions for N = 3 and N = 5 at the R´enyi entropies plane H1 and Hq: q = 2 (a,b); q = 3 (c,d) and q = 4 (e,f). Thin dotted lines in each panel represent the monotonicity lower bounds (9) while bold dotted curves in panel (a) and (b) denote upper bound (16). Let us emphasize in this point that results presented in [15] do not close the issue of finding bounds and relations between different entropies. Results analogous to (18) for a more general class of entropy functions were recently obtained by Berry
Page 7
R´ENYI EXTRAPOLATION OF SHANNON ENTROPY 7 and Sanders [17]. A more precise lower bound for Shannon entropy, quadaratic in terms of index of coincidence (purity) was found by Topsøe [18]. 4. N–dependent bounds for Shannon entropy 4.1. Bounds based on H2 and the length N of the probability vector. Results (18) allow us to obtain bounds for a value of the entropy Hq, provided the value Hs is known. Let us first assume that the entropy H2 is known and we want to extract some information on H1. We start computing the R´enyi entropy of order two, H2(Q1,N(a)) = −ln h(1 + (N −1)a)2 N 2
- (N −1)(1 −a)2 N 2 i (19) and invert it to obtain a = r N exp(−H2) −1 N −1 . (20) In this way we receive sharp upper bounds for q ∈(0, 2) Hq(P) ≤ 1 1 −q ln h1 + (N −1)a N q
- (N −1) 1 −a N qi , (21) which for q →1 reduces to H1(P) ≤Hu 12 := (1 −N)1 −a N ln 1 −a N −1 + a(N −1) N ln 1 + a(N −1) N , (22) with a given by (20). To obtain analogous lower bound we find k such that ln(k −1) ≤H2 ≤ln k and compute H2(Qk−1,k(a)). Also this relation may be easily inverted providing a = p k(k −1) exp(−H2) + 1 −k. Thus we arrive at a lower bound for the R´enyi entropy Hq(P) ≥ 1 1 −q ln h (k −1)zq + yqi , (23) and in particular case, for the Shannon entropy H1(P) ≥Hd 12 := (1 −k)z ln z −y ln y , (24) with z = (k + a −1)/(k2 −k) and y = (1 −a)/k. 4.2. Bounds based on H3 and N. Let us now assume, we know the value of the R´enyi entropy H3. As in (19) we compute H3(Q1,N(a)), and invert it finding a ∈[0, 1] as the largest (real) root of the polynomial Wu(a) = a3 + a2 3 N −2 −N 2 exp(−2H3) −1 (N −1)(N −2) = 0. (25) Then the upper bound valid for q ∈(0, 3) is given by the same formula (21) with a given by the root of (25) instead of (20). For q →1 one obtains then the upper bound for the Shannon entropy H1(P) ≤Hu 13 := (1 −N)1 −a N ln 1 −a N −1 + a(N −1) N ln 1 + a(N −1) N , (26) with a determined by (25).
Page 8
8 K. ˙ZYCZKOWSKI To get the lower bound we look for k′ such that ln(k′ −1) ≤H3 ≤ln k′. The relation H3(Qk′−1,k′(a)) may be inverted explicitly for k′ = 2 providing a = p [4 exp(−2H3) −1]/3. For k′ > 2 a is given by the only root of the polynomial Wd(a) = a3 + a2 3(k′ −1) 2 −k′
- (k′ −1)2 2 −k′ [1 −k′2 exp(−2H3)] = 0. (27) in the interval [0, 1]. The lower bound for the R´enyi entropies has the same form as (23) and gives for the Shannon entropy H1(P) ≥Hd 13 := (1 −k′)z′ ln z′ −y′ ln y′ (28) with z′ = (k′ + a −1)/(k′(k′ −1)) and y′ = (1 −a)/k′.
- Combined extrapolation In previous section we obtained two upper bounds for the Shannon entropy: (23) stemming from the R´enyi entropy H2, and (12) obtained from H3. The latter is in general a worse one3, but it allows for a linear extrapolation, which gives Hup(⃗x) := 2Hu 12(⃗x) −Hu 13(⃗x). (29) Our numerical results allow us to advance the following Conjecture. For any probability distribution ⃗x the bound H1(⃗x) ≤Hup(⃗x) (30) holds. In the same way one may try to extrapolate lower bounds defining Hd(⃗x) := 2Hd 12(⃗x) −Hd 13(⃗x). For certain probability vectors this quantity may give a useful approximation for the Shannon entropy. Interestingly, a relation analogous to (30), H1(⃗x) ≥Hd(⃗x) is not true: it is violated e.g. if k ̸= k′. Making use of the rigorous bound (12) and the conjecture (30) we may suggest to estimate the unknown value of the Shannon entropy by the mean value H′ ∗(⃗x) := 1 2[Hup(⃗x) + Hd23(⃗x)] = Hu 12(⃗x) + H2(⃗x) −1 2[Hu 13(⃗x) + H3(⃗x)], (31) which is obtained by combined methods based as well on the lower as well as on the upper bounds. Observe that this explicitly computable quantity involves only R´enyi entropies H2 and H3 and the dimension N. This estimation may be improved noting that the rigorous lower bounds (12) and (24) are not equivalent. Since for some distributions the latter bound gives better (higher) results, we may improve (31) writing H∗(⃗x) := 1 2Hu(⃗x) + 1 2max{Hd23(⃗x), Hd 12(⃗x)}. (32) If the value of the zero-entropy, H0, is known, one may replace the upper bound Hu(⃗x) used above by the minimum, min{Hu0(⃗x), Hup(⃗x)}. Figure 3 shows the bounds and the estimations described above for a randomly chosen probability distributions with N = 15 components. The overall quality of the proposed estimations for the Shannon entropy may be judged from Fig. 4, which shows the histogram of the deviations δ1 = H∗−H1 and δ2 = Hd −H1 for a sample 3Knowing the function H(q) at q = 3 we have less information on H1, than knowing it at q = 2.
Page 9
R´ENYI EXTRAPOLATION OF SHANNON ENTROPY 9 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.5 1 1.5 2 2.5 N=15 H q q Figure 3. R´enyi entropies Hq for a random probability distribu- tion of size N = 15 (solid line). Dense dotted curves represent H2 bounds: upper (21) and lower (23), while faint dotted curve denote analogous bounds based on H3. Straight dashed–dotted lines show lower (12) and upper (29) extrapolations. The exact value of H1 is denoted by (∗), while the estimation (32) by the middle (×). of 104 probability vectors generated randomly according to the statistical (Fisher– Rao) measure on the N −1 dimensional simplex [19]. This measure has a simple geometric interpretation: is suffices to consider a unit vector ⃗t distributed uniformly on the sphere SN−1, and to define the probability vector ⃗x = {t2 1, …, t2 N}. Numerical results obtained in this way allow us to conclude that the proposed estimation (32) provides a useful all-purpose approximation of the Shannon entropy. Note that the precision of this approximation decreases with the length N of the probability vector. To judge about possible application of the estimate H∗in the analysis of physical data, one should perform analogous numerical simulations with random vectors ⃗x generated according to a specific probability distribution adjusted to a given physical problem. 6. Concluding Remarks In this work we considered the problem of finding the bounds and extrapola- tions for the Shannon (entropy), provided some of the Renyi entropies are known. In general, generalized entropies of integer order q = 2 and q = 3 are easiest to calculate, and they are sufficient to obtain bounds (10) and (12) for the Shannon entropy. The quality of the bound may be improved if the length N of the prob- ability vector is known. Then an explicit extrapolation (32) allows us to estimate the actual value of the entropy H.
Page 10
10 K. ˙ZYCZKOWSKI −0.3 −0.25 −0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15 0 2 4 6 8 10 12 14 16 P δ Figure 4. Histogram of deviations of the extrapolations form the real value of the Shannon entropy H1: (∗) denotes the probability density of error δ1 = H∗−H1 of the estimation (32), while (o) denotes results for the error δ2 = Hd −H1 of the lower bound (12) for a sample of 104 random probability vectors of size N = 10. Note that the bounds and extrapolations discussed may be easily rewritten in terms of a non-extensive entropy Sq(⃗x) := 1 1 −q h n X i=1 xq i −1 i (33) used by Havrda and Charvat [20] and Daroczy [21], which became often used in statistical physics after the seminal work of Tsallis [22]. In particular, the linear entropy L is just the nonextensive entropy of order two, L(⃗x) = S2(⃗x), and the bounds between Hq and H1 imply analogous relations between Sq and H1. In fact the plot presented in [13] shows bounds between von Neumann entropy and the linear entropy and they follow directly from relation (18) proven in [15]. The issue of comparing the R´enyi entropies H0, H1 and H2 is closely related to the problem of describing the degree of chaos of an analyzed classical dynamical system by the topological entropy K0, the Kolmogorov–Sinai (metric) entropy K1 and the correlation entropy K2. These dynamical entropies are defined as the rate of the increase of the R´enyi entropies in time [16], but since the length of the probability vector is not finite the N–dependent bounds discussed in this work are not applicable. The same concerns comparison of generalized fractal dimensions Dq of fractal measures: the box–counting dimension D0, the information dimension D1 and the correlation dimension D2, which also form a non–increasing function of the R´enyi parameter q [16], are defined by the limit N →∞. The generalized entropies may also be used to characterize localization properties of continuous probability distributions. For instance, any pure quantum state may be represented in the phase space by the Husimi distribution. Its localization
Page 11
R´ENYI EXTRAPOLATION OF SHANNON ENTROPY 11 can be measured by the Wehrl entropy defined as the continuous (Boltzmann– Gibbs) entropy of the Husimi distribution [23]. In an analogous way one may define generalized R´enyi–Wehrl entropies [24, 11], and above results may be used to obtain similar bounds for the Wehrl entropy. It is a pleasure to thank P. Harremo¨es and F. Topsøe for explaining us their results prior to publication and several fruitful comments. I am also thankful to I. Bengtsson, A.Bia las, A. Ostruszka, F. Mintert, W. Munro and W. S lomczy´nski for inspiring discussions and D. Berry, B. Sanders and I. Varga for helpful corre- spondence. Financial support by Komitet Bada´n Naukowych in Warsaw under the grant 2P03B-072 19 and by a research grant of the Volkswagen Stiftung is gratefully acknowledged. 7. Corrigendum – February 17, 2005 Eq. (8) implies that (1 −q)Hq is a convex function of q. However, it does not imply that the R´enyi entropy Hq is a convex function of q. For instance, the R´enyi entropy for a N = 20 probability vector P = 1 100(43, 3, … , 3) is not a convex function of q. I am deeply obliged to Christian Schaffner for drawing my attention to this fact. Therefore, equations (10), (11), and (12) are not satisfied and conjecture (30) cannot hold. Furthermore, equations (13), (31), (32) may only be used to extrapo- late the unknown value of the Shannon entropy, from the available data on H2 and H3. On the other hand, we would like to mention that the lack of convexity of Hq in q does not influence the bounds between particular values of the R´enyi entropy presented in sections 3 and 4 of the present paper. References
- C. Shannon, Mathematical theory of communication, Bell System Tech. J. 27, 379 (1948)
- A. I. Khinchin, Mathematical foundations of information theory, Dover, 1957,
- A. R´enyi, On measures of entropy and information, in. Proc. Fourth. Berkeley Symp. Math. Stat. Prob. 1960, Vol. I, p.547, (University of California Press, Berkeley, 1961).
- J. N. Kapur, Measures of Information and Their Applications (John Wiley & Sons, New York, 1994).
- C. Arndt. Information Measures: Information and its Description in Science and Engineering (Springer, Berlin 2001).
- P. Horodecki and A. Ekert, Direct detection of quantum entanglement, Phys. Rev. Lett. 89, 127902 (2002).
- C. M. Alves, P. Horodecki, D. K. L. Oi, L. C. Kwek, and A. K. Ekert, Direct estimation of functionals of density operators by local operations and classical communication, preprint quant-ph/0304123
- A. Bia las and W. Czy˙z, Renyi Entropies in Multiparticle Production, Acta Phys. Pol. B 31, 2803 (2000).
- A. Bia las, W. Czy˙z and A. Ostruszka, Renyi entropies in particle cascades, Acta Phys. Pol. B 34, 69 (2003).
- J. Pipek and I. Varga, Universal scheme for the spacial–localization properties of one–particle states in finite d–dimensional systems, Phys. Rev. A 46, 3148 (1992).
- I. Varga and J. Pipek, On R´enyi entropies characterizing the shape and the extension of the phase space representation of quantum wave functions in disordered systems, arXiv preprint cond-mat/0204041 (2002).
- K. ˙Zyczkowski, I. Bengtsson, Relativity of pure states entanglement, Ann. Phys. (N.Y.) 295, 115-135 (2002)
Page 12
12 K. ˙ZYCZKOWSKI 13. T.-C. Wei, K. Nemoto, P.M. Goldbart, P.G. Kwiat, W.J. Munro and F. Verstraete, Maximal entanglement versus entropy for mixed quantum states, Phys. Rev. A 67, 022110 (2003) 14. F. Topsøe, Some inequalities for information divergence and Related measures of discrimination IEEE Trans. Inform. Theory 46, 1602 (2000). 15. P. Harremo¨es and F. Topsøe, Inequalities between Entropy and Index of Coinci- dence derived from Information Diagrams, IEEE Trans. Inform. Theory 47, 2944-2960 (2001) 16. C. Beck and F. Schl¨ogl, Thermodynamics of chaotic systems, (Cambridge University Press, Cambridge, 1993). 17. D. W. Berry and B. C. Sanders, Bounds on entropy, preprint arXiv quant-ph/0305059, (2003) 18. F. Topsøe, Entropy and Index of Coincidence, lower bounds, preprint, Copenhagen, 2003 19. R. A. Fisher, Theory of Statistical Estimation Proc. Cambridge Philos. Soc. 22, 700 (1925). 20. J. Havrda and F. Charvat, Quantification methods of classification Processes: Con- cept of structural α–Entropy, Kybernetica 3, 30 (1967) 21. Z. Daroczy, Inf. Control 16, 36 (1970) 22. C. Tsallis, Possible generalization of Boltzmann–Gibs statistics, J. Stat. Phys. 52, 479 (1988) 23. A. Wehrl, General properties of entropy, Rev. Mod. Phys. 50, 221 (1978) 24. S. Gnutzmann and K. ˙Zyczkowski, Renyi-Wehrl entropies as measures of localization in phase space, J. Phys. A 34, 10123-10139 (2001). Instytut Fizyki im. Smoluchowskiego, Uniwersytet Jagiello´nski, ul. Reymonta 4, 30- 059 Krak´ow, Poland, and, Centrum Fizyki Teoretycznej, Polska Akademia Nauk, Al. Lotnik´ow 32/44, 02-668 Warszawa, Poland E-mail address: karol@tatry.if.uj.edu.pl
Canonical Hub: CANONICAL_INDEX